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(54) Method and apparatus for instrument tracking on a scrolling series of 2D fluoroscopic 
images 



(57) The present invention relates to a method (200) 
and apparatus for instrument tracking on a series of im- 
ages. The method relates to performing instrument 
tracking on an image. The method comprises collecting 
(210) at least one image and computing (230) at least 
one of a position and orientation of at least one instru- 
ment for the at least one collected image. The method 
further relates to displaying (240) at least one, some or 
all of the collected and computed data either separately 
or in any combination. 
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Description 

[0001] The present invention relates to instrument 
tracking. More specifically, at least one embodiment of 
the present invention relates to instrument tracking on 5 
a scrolling series of images using a display apparatus. 
[0002] Computer assisted surgical procedures includ- 
ing instrument navigation are known. One common 
component of such computer assisted surgical proce- 
dures includes instrument navigation as one method for 10 
transferring knowledge of the instrument position and/ 
or orientation to the user or operator on an ongoing ba- 
sis. Traditionally, such knowledge transfer may be ac- 
complished by acquiring static data via an imaging mo- 
dality that depicts a patient's anatomy (or portion there- 15 
of). The image of an instrument may be displayed on 
such static data. This display may include, for example, 
multiple 2D fluoroscopic images, multiple 2D slices 
through 3D data, a 3D surface model, or other data. 
[0003] It should be appreciated that such prior known 20 
computer assisted surgical procedures are limited in the 
manner in which the instrument position information is 
perceived or understood by the user. For example, in a 
computer assisted surgical procedure using 2D or 3D- 
slice planar images, the user pieces together a mental 25 
picture of the patient anatomy from the limited and dis- 
tributed information provided during the procedure. 
[0004] In a computer assisted surgical procedure us- 
ing a 3D model, for example, the user typically looks at 
one particular projection of the model at a time. Looking 30 
at one projection at a time may hide significant data (in- 
cluding parts of the instrument) or require the removal 
of significant sections of the model in order to view the 
desired anatomy, all of which generally requires signifi- 
cant user interaction and visualization skills to under- 35 
stand the provide instrument positional information. 
[0005] One embodiment of the present invention re- 
lates to a method of performing instrument tracking on 
an image. The method comprises collecting at least one 
image and computing at least one of a position and ori- 40 
entation of at least one instrument for the at least one 
collected image. The method further comprises display- 
ing the at least one collected image, the at least one of 
a position and orientation of the at least one instrument 
and/or at least one image of the at least one instrument ^ 
located at said at least one of a position and orientation. 
[0006] In at least one embodiment, the method further 
comprises collecting at least one of a position and ori- 
entation of at least one collected image and/or at least 
one instrument using at least one measurement device so 
or measurement method. The method may be used in 
computing the at least one position and/or orientation of 
the at least one instrument for the at least one collected 
image. It is further contemplated that each of the meas- 
urement devices may use one or more appropriate tech- 55 
nologies, including but not limited to electromagnetic, 
optical, laser, and physical measurement such as via 
encoder feedback. It is further contemplated that each 



of the measurement methods may use one or more al- 
gorithmic methods, including but not limited to positional 
prediction methods that may or may hot use phantom 
data, and registration methods that may or may not be 
entirely image-based. 

[0007] Methods are contemplated which comprise 
collecting at least a plurality of 2D fluoroscopic images 
and continuously scrolling through the plurality of col- 
lected images using a display. Methods are also con- 
templated which comprise projecting the at least one po- 
sition and orientation of the at least one instrument into 
the plurality of collected images in sequence. 
[0008] In one or more embodiments the method com- 
prises calibrating the at least one collected image such 
that the at least one position and orientation of the at 
least one image may be accurately displayed. In at least 
this embodiment, the method comprises selecting at 
least one calibrated image to be the current image. The 
method further comprises computing the at least one 
position and orientation for the at least one instrument 
for the current image. 

[0009] Methods are further contemplated for using 
collected and/or computed measurement data of at 
least one of a current or past nature, from real or simu- 
lated sources, in order to compute or recompute the at 
least one position and/or orientation for the at least one 
instrument for any, some, or all of the at least one col- 
lected images, including the current image. 
[0010] In still other embodiments, the method com- 
prises collecting the at least one image using at least 
one image collection device that moves or is moveable. 
It is further contemplated that this device may be at least 
one C-arm which may or may not be coupled with at 
least one measurement device or measurement meth- 
od.. 

[0011] Still another embodiment of the present inven- 
tion relates to a method of performing instrument track- 
ing on a series of images using a display device. This 
embodiment may comprise collecting a series of 2D im- 
ages. The series of 2D images may be calibrated such 
that at least one of a position and orientation of at least 
one instrument may be accurately displayed in at least 
one image of the series of 2D images. This method may 
further comprise selecting at least one image of the se- 
ries of 2D images to be the current image and computing 
the at least one position and orientation of the at least 
one instrument for the current image. The at least one 
position and orientation may be projected into the cur- 
rent image, which may then be displayed. 
[0012] Other embodiments of the present invention 
comprise collecting the series of 2D images (a series of 
2D fluoroscopic images for example) using an imaging 
device that moves or is moveable, where this device is 
a C-arm. For example, contemplated methods include 
continually scrolling through the series of images in a 
display, wherein at least one position and orientation of 
at least one instrument are projected into at least one 
image of the series of images. It is further contemplated 
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that at least the current image may be incremented, 
wherein incrementing the current image comprises se- 
lecting a different collected image to be the current im- 
age based on an algorithm and recomputing at least one 
of the position and orientation of at least one instrument 
for the new current image. 

[001 3] Still another embodiment of the present inven- 
tion relates to a system or apparatus for performing in- 
strument tracking on a series of images. At least one 
embodiment of the apparatus comprises at least one 
collection device that moves or is moveable, at least one 
processing device and an output. In at least one embod- 
iment, the collection device is adapted to collect the set 
of images. The processing device communicates with 
at least the collection device that moves or is moveable 
and is adapted to produce a scrolling series of images, 
at least one of the scrolling series of images including 
at least one of a position and orientation of at least one 
instrument. The output communicates with at least the 
processing device and is adapted to display at least one 
image of the scrolling series of images. 
[0014] In at least one embodiment of the apparatus, 
the moveable collection device comprises at least one 
C-arm. Further, the moveable collection device compris- 
es at least transmitter and detector devices. In at least 
one embodiment, the transmitter and detector devices 
may be used to measure the relative position and/or 
change in position of one or more components of the 
collection device with respect to objects seen within the 
image or images, and/or to measure the relative position 
and/or change in position of any instrumentation that 
may be present or that may be introduced into the area 
around the imaged objects before, during, or after the 
time when the moveable collection device is present in 
the area around the imaged objects. 
[0015] Embodiments of the invention will now be de- 
scribed; by way of example, with reference to the ac- 
companying drawings, in which: 

FIG. 1 illustrates an imaging or display system, de- 
vice, machine or apparatus used in accordance with 
certain embodiments of the present invention. 

FIG. 2 illustrates a high-level flow diagram of a 
method of performing instrument tracking using an 
imaging or display apparatus similar to that depict- 
ed in FIG. 1 in accordance with certain embodi- 
ments of the present invention. 

FIG. 3 illustrates a detailed flow diagram depicting 
a method of performing instrument tracking on a 
scrolling series of fluoroscopic images using an im- 
aging or display apparatus similar to that depicted 
in FIG. 1 in accordance with certain embodiments 
of the present invention. 

[0016] The foregoing summary, as well as the follow- 
ing detailed description of certain embodiments of the 



present invention, will be better understood when read 
in conjunction with the appended drawings. For the pur- 
pose of illustrating the invention, certain embodiments 
are shown in the drawings. It should be understood, 
5 however, that the present invention is not limited to the 
arrangements and instrumentality shown in the at- 
tached drawings. 

[0017] For the purpose of illustration only, the follow- 
ing detailed description references a certain embodi- 
10 ment of an imaging or display system, machine, appa- 
ratus or device. However, it is understood that the 
present invention may be used with other devices or im- 
aging systems. 

[001 8] At least one embodiment of the present inven- 
ts tion relates to instrument tracking. More specifically, at 
least one embodiment of the present invention relates 
to instrument tracking on a scrolling series of images 
using a display apparatus. 

[001 9] At least one embodiment of the present inven- 
20 tion improves the method of transferring instrument po- 
sition and/or orientation information to a user, making 
such information easierto use and understand. It should 
be appreciated that making such information easier to 
use and understand may improve efficiency and/or ease 
25 of use of surgical workflow and may increase product 
quality perception. One or more embodiments uses nat- 
ural human ability to perceive 3D information from ani- 
mated 2D data to transfer the position and/or orientation 
of a surgical instrument to the user. Previous designs 
30 have used static, non-animated data to accomplish such 
transfer in a method that requires substantial learned 
skill. 

[0020] In humans, mentally perceiving a 3D volume 
occurs naturally when motion is introduced to a static 

35 set of data, assuming that the static set of data is suit- 
able for animation. One or more embodiments of the 
present invention collects one or more (i.e., a series) of 
images (2D fluoroscopic images for example) using a 
movable collection device (a C-arm for example al- 

40 though other moveable collecting devices are contem- 
plated) adapted to be rotated about a patient's anatomy 
of interest. It should be appreciated that, in at least one 
embodiment, this collected series of images or data set 
is appropriate for animation. 

45 [0021 ] One embodiment provides feedback to the us- 
ers using such data set, where the data set is continu- 
ously scrolled in a window or display, and further where- 
in an instrument's position and/or orientation is project- 
ed onto or into one or more images. In one embodiment, 

50 the instrument position and/or orientations are projected 
into or onto each image in sequence. Such scrolling of 
the data set, including the position and/or orientation of 
their instrument, enables users to understand the instru- 
ment position and/or orientation information in 3D. This 

55 takes advantage of the user's natural abilities to per- 
ceive 3D volume, thus eliminating learning new skills to 
understand the data. In addition, at least one embodi- 
ment does not require direct user interaction to optimize 
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the display as required by one or more known methods 
of the computer assisted surgical procedures. 
[0022] In at least one embodiment, the instrument 
may be fully displayed on one or more images of the 
data set without partial hiding data due to depth, as is 
required for 3D model instrumentation representations. 
This is possible as the depth perception is naturally con- 
veyed to the user by the animation, and the position in- 
formation may be fully contained within a single visual- 
ization window. This improves ease-of-use, surgical 
workflow and product quality perception. At least one 
embodiment may comprise collecting at least one of a 
position and orientation of at least one collected image 
and/or at least one instrument using at least one meas- 
urement device or measurement method. The measure- 
ment method may be used in computing the at least one 
position and/or orientation of the at least one instrument 
for the at least one collected image. It is further contem- 
plated that each of the measurement devices may use 
one or more appropriate technologies, including but not 
limited to electromagnetic, optical, laser, and physical 
measurement such as via encoderfeedback. It is further 
contemplated that each of the measurement methods 
may use one or more algorithmic methods, including but 
not limited to positional prediction methods that may or 
may not use phantom data, and registration methods 
that may or may not be entirely image-based. 
[0023] In at least one embodiment measurement data 
of at least one of a current and past nature may be col- 
lected and/or computed, from real or simulated sources, 
in order to compute or recompute the at least one posi- 
tion and/or orientation for the at least one instrument for 
any, some, or all of the at least one collected images, 
including the current image. 

[0024] FIG. 1 illustrates an imaging system, device, 
machine or apparatus, generally designated 100, used 
in accordance with at least one embodiment of the 
present invention. It is contemplated that system 100 
may comprise a variety of systems or combinations 
thereof, including an X-ray system, a CT system, an 
EBT system, an ultrasound system, an MR system, an 
electromagnetic (EM) tracking system, optical tracking 
system, laser tracking system, encoder feedback sys- 
tem, algorithmic position prediction system (that may or 
may not use phantom data), algorithmic registration sys- 
tem (that may or may not be entirely image-based), or 
other imaging or positional measurement system. 
[0025] In at least one embodiment, system 100 in- 
cludes a C-arm 110, one or more X-ray sources 120, 
one or more X-ray detectors 130, one or more electro- 
magnetic (EM) sensors 1 40, at least one EM transmitter 
150, an image processing computer or device 160, a 
tracker module 170, a positioning device 180, and an 
output 1 90. In the illustrated embodiment, tracker mod- 
ule 1 70 is depicted communicating with at least EM sen- 
sor 140, EM transmitter 150, and image processing 
computer 160. FIG. 1 further illustrates image process- 
ing computer 1 60 communicating with at least X-ray de- 



tector 130, tracker module 170 and output 190. While 
the above described interconnections are illustrated, 
other interconnections are contemplated. 
[0026] In at least one embodiment, X-ray source 120 

s and X-ray detector 130 are mounted on opposing sides 
of C-arm 1 1 0, where X-ray source 1 20 and X-ray detec- 
tor 1 30 may be movably mounted on the C-arm 110. In 
one embodiment, EM sensor 140 is mounted on X-ray 
detector 130. The EM transmitter 150 is positioned on 

10 or proximate an object 112 (i.e., a patient) to be imaged. 
Alternatively, EM transmitter 150 may be located on the 
X-ray detector 130, and EM sensor 140 may be located 
on object or patient 1 1 2 being imaged. The object or pa- 
tient 112 is positioned on or in positioning device 180. 

15 in at least one embodiment, positioning device 1 80 com- 
prises a table, a table bucky. a vertical bucky, a support 
or other positioning device adapted to be used with the 
present invention. 

[0027] In at least one embodiment, C-arm 110 is 
20 adapted to be moved in several directions along multiple 
image acquisition paths, including, for example, an or- 
bital direction, a longitudinal direction, a lateral direction, 
a transverse direction, a pivotal direction and a "wig- 
wag" direction (where at least one example of such 
25 movement is indicated by the dashed lines in FIG. 1 ). In 
at least one embodiment, X-ray source 1 20 and detector 
130 are movably positioned on C-arm 110 (where such 
moved is indicated by the dashed lines). Thus, the C- 
arm 110, along with X-ray source 120 and X-ray detector 
30 1 30, may be moved and positioned about the position- 
ing device 180 on or in which object 112 has been situ- 
ated. 

[0028] The C-arm 110 is used to position the X-ray 
source 120 and detector 130 about object 112 so that 

35 one or more X-rays 1 05 (or other energy) may irradiate 
object 112 to produce one or more images. The C-arm 
1 1 0 may be moved or re-positioned at a variety of scan 
angles around object 112, obtaining a plurality of imag- 
es. As the C-arm 110 moves, the distance between the 

40 X-ray detector 130 and the object 112 may vary. Further 
the distance between X-ray source 120 and object 112 
may also vary. 

[0029] It is contemplated that, in at least one embod- 
iment, X-ray source 120 and detector 130 on C-arm 110 

45 may move in a cross-arm or orbital motion, for example. 
In an orbital motion, the X-ray source 120 and the de- 
tector 1 30 do not move in a circular path. In tomographic 
image reconstruction using orbital motion, a distance 
between Xray detector 130 and object 112 (and a dis- 

50 tance between source 120 and object 112) may vary 
during collection of projection images. 
[0030] In at least one embodiment, a position of the 
X-ray detector 1 30 may be recorded for one or more pro- 
jection images. Additionally, the distance between de- 

55 tector 1 30 and the X-ray source 1 20 may be determined. 
A magnification change may be quantified and compen- 
sated for during image reconstruction using the position 
of detector 130 and the detector-to-object distance. The 
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EM sensor 140 or other tracking device may be placed 
on detector 130. The EM transmitter 150 or other track- 
ing device may be placed on the object 112. Data from 
the sensor 140 and transmitter 150 may be used to de- 
termine a position of detector 1 30 during a trajectory of s 
detector 130. Other tracking devices, such as optical or 
mechanical tracking devices, may be used to determine 
a position of one or more components in the system 1 00. 
[0031] In at least one embodiment, transmitter 150 
broadcasts a signal, such as a magnetic field, that is 
detected by sensor 140. The tracker module 170 may 
use data from the transmitter 1 50 to determine a position 
of the detector 1 30 with respect to object 1 1 2. Differenc- 
es in position and, thus, distance between the detector 
130 and the object 112 correspond to differences in 
magnification in obtained X-ray projection images. 
[0032] Changing the distance between detector 130 
and object 112 and/or the distance between the source 
1 20 and object 1 1 2 changes the magnification of the ob- 
ject projected onto the detector for point sources or 
near-point sources that emit non-parallel beams, such 
as X-rays. If the field of view of the X-ray source 120 is 
constant, as an object 1 1 2 approaches the X-ray source 
120 : the object 112 occupies more of the field of view 
and therefore projects a larger image onto the detector 
130. In an embodiment, the detector-to-object distance 
may be varied to maintain the object 112 at a virtual iso- 
center of the system 100. In an embodiment, the C-arm 
110 and/or the source 120 and/or detector 130 on the 
C-arm 110 may be moved in any plane or not moved to 
position the object 1 1 2 at the virtual isocenter in the field 
of view of the detector 1 30. Measurement of the varying 
detector-to-object and/or source-to-object distance en- 
ables the image processor 160 to compensate for the 
change in distance and thus the change in magnifica- 
tion. The tracker module 1 70 may use data from the EM 
sensor 1 40 and EM transmitter 1 50 or other tracking de- 
vice to track the detector-to-object distance. 
[0033] Alternatively, EM sensor 1 40 or EM transmitter 
150 may be mounted on the source 120 with the EM 
transmitter 150 or EM sensor 140 on the object to de- 
termine position of the source 1 20. A position of the X- 
ray source 120 may be recorded and used with the 
source-to-detector distance to determine and account 
for the magnification change. Tracker module 170 may 
monitor a position and/or orientation of an instrument or 
tool used during a diagnostic or surgical procedure, for 
example. 

[0034] The tracker module 1 70 monitors a position of 
at least object 112, X-ray detector 130. and/or X-ray 
source 120 for example. The tracker module 170 may 
provide position data in a reference coordinate system 
with respect to at least object 112, source 120, and/or 
detector 1 30. The image processor 1 60 may use the po- 
sition data when processing the image data to recon- 
struct 2D and/or 3D images. The position data may also 
be used for other purposes, such as surgical navigation, 
for example. In one embodiment, the tracker module 



170 calculates the positions of the X-ray detector 130 
and object 112 with respect to a coordinate system de- 
fined relative to a coordinate system reference point or 
central axis (on a continuous basis for example). In at 
least one embodiment, the image processor 1 60 gener- 
ates control or trigger commands to the X-ray source 
1 20 or source controller to scan the object based on po- 
sition data. 

[0035] In at least one embodiment, the image proces- 
sor 160 collects a series of image exposures from the 
detector 130 as the C-arm 110 is moved. The detector 
130 receives an image exposure each time the X-ray 
source 120 is triggered. The image processor 160 com- 
bines image exposures with reference data, recon- 
structing a 3D volumetric data set for example. The 3D 
volumetric data set may be used to generate images, 
such as slices, or a region of interest from the object 
112. For example, the image processor 160 may pro- 
duce sagittal, coronal, and/or axial views of a patient 
spine, knee, or other area from the volumetric data sets. 
The image processor 160 may be implemented in soft- 
ware and/or hardware, where the image processor 160 
may comprise a general purpose computer, a micro- 
processor, a microcontroller, and/or an application-spe- 
cific integrated circuit, for example. 
[0036] In one or more embodiments, 3D image recon- 
struction may be formed by combining successive slices 
or scanned planes of object 112 using a fan beam for 
example. A 3D image reconstruction may also be 
formed by rotating source 120 and detector 130 around 
object 112 to obtain cone or area beam projections of 
the object. In a cone beam projection, the object may 
be illuminated with a point source and X-ray flux meas- 
ured on a plane by the detector 130. The distance from 
object 1 1 2 to the detector 1 30 and the distance from ob- 
ject 112 to the source 120 may be used to determine 
parallel projections for image reconstruction. 
[0037] Filtered backprojection may also be used to re- 
construct a 3D image based on filtering and backpro- 
jecting a plane in a cone beam. In a filtered backprojec- 
tion, individual fan beam or cone beam projections are 
analyzed and combined to form a 3D reconstruction im- 
age. Fan beams are tilted out of a source-detector plane 
of rotation for analysis in a new coordinate system for 
filtered backprojection. Projection data is weighted 
based on distance and convolved. Then, the convolved 
weighted projections are backprojected over a 3D re- 
construction grid to reconstruct a 3D image. 
[0038] After the one or more images have been re- 
constructed, the image processor 160 may transmit the 
one or more image(s) to output 190. It is contemplated 
that output 1 90 may comprise a display, a printer, fac- 
simile, electronic mail, a storage unit, or other medium, 
for example. It is further contemplated that, in at least 
one embodiment, output 190 may comprise a laptop, 
PDA, cell phone or other device wirelessly communicat- 
ing with image processor computer 1 60. The image may 
be displayed and/or stored via output 190 for use by a 



15 



20 



25 



30 



35 



40 



45 



50 



5 



9 



EP 1 561 423 A1 



10 



user such as a technician, physician, surgeon, other 
healthcare practitioner, or security officer. 
[0039] In operation, for example, a patient's mid-spi- 
nal area may be scanned in the system 1 00. The C-arm 
110 may not reach all positions of a mid-spinal scan 5 
when the patient is positioned on a table (i.e., positioner 
1 80). Therefore, the C-arm 1 1 0 may be moved and po- 
sitioned from a side. As the C-arm 110 is moved in a 
non-circular motion, the spine may not remain centered 
in scanned images because the path of the C-arm 110 10 
may not be circular. The C-arm 1 1 0 may be moved, such 
as by raising and lowering the C-arm 110 on a C-arm 
support, to keep the spine in the center (e.g., a virtual 
isocenter). As the C-arm 11 0 is moved and the spine is 
not moved, the spine is located closer to or farther from 15 
X-ray source 120. Thus, obtained images may have a 
different magnification from start to finish (for example, 
five vertebral levels in a first image to three vertebral 
levels in a last image due to more magnification) be- 
cause the C-arm 1 1 0 moves in a non-circular arc. 20 
[0040] A change in magnification may be determined 
because the position of the detector 130 with respect to 
the object being scanned is measured by the tracker 
module 170 using the EM transmitter 150 and sensor 
1 40, for example. The magnification change may be tak- 25 
en into account during reconstruction of a 3D volume 
image of the mid-spinal area. Rather than using a fixed 
distance in standard image reconstruction algorithms, 
the variable distance values are used in reconstruction 
calculations for the image(s). 30 
[0041] In at least one embodiment, the transmitter 
and detector devices may be used to measure the rel- 
ative position and/or change in position of one or more 
components of the collection device with respect to ob- 
jects seen within the image or images, and/or to meas- 35 
ure the relative position and/or change in position of any 
instrumentation that may be present or that may be in- 
troduced into the area around the imaged objects be- 
fore, during, or after the time when the moveable collec- 
tion device is present in the area around the imaged ob- 40 
jects. 

[0042] As provided previously, at least one embodi- 
ment of the present invention improves the methods of 
transferring instrument position and/or orientation infor- 
mation to a user, making such information easier to use 45 
and understand. It should be appreciated that making 
such information easier to use and understand may im- 
prove efficiency and/or ease of use of surgical workflow 
and may increase product quality perception. At least 
one or more embodiments uses natural human ability to so 
perceive 3D information from animated 2D data to trans- 
fer the position and/or orientation of a surgical instru- 
ment to a user. 

[0043] Perceiving a 3D volume occurs naturally for 
humans when motion is introduced to a static set of data. 55 
One or more embodiments of the present invention col- 
lects one or more (i.e., a series) of images (2D fluoro- 
scopic images for example) using a movable collection 



device (a C-arm for example) adapted to be rotated 
about a patient's anatomy of interest. It should be ap- 
preciated that, in at least one embodiment, this collected 
series of images or data set is appropriate for animation. 
[0044] One embodiment provides user feedback, 
where the data set is continuously scrolled in a window 
or display. Further an instrument's position and/or ori- 
entation may be projected onto or into one or more im- 
ages. In one embodiment, the instrument position and/ 
or orientations projected into each image in sequence. 
Such scrolling of the data set, including the position and/ 
or orientation of their instrument, enables the user to un- 
derstand the instrument position and/or orientation in- 
formation in 3D, thus taking advantage of the user's nat- 
ural abilities to perceive 3D volume, thus eliminating 
learning new skills to understand the data. In addition, 
at least one embodiment does not require direct user 
interaction to optimize the display as required by one or 
more known methods of the computer assisted surgical 
procedures. 

[0045] In at least one embodiment, the instrument 
may be fully displayed on one or more images of the 
data set without partial hiding data due to depth, as is 
required for 3D model instrumentation representations, 
since the depth perception is naturally conveyed to the 
user by the animation, and the position information may 
be fully contained within a single visualization window. 
This improves ease-of-use, surgical workflow and prod- 
uct quality perception. 

[0046] FIG. 2 illustrates a high-level flow diagram de- 
picting a method, generally designated 200, for perform- 
ing instrument tracking in accordance with certain em- 
bodiments of the present invention. In at least one em- 
bodiment, method 200 may perform such instrument 
tracking using an imaging system. In at least one em- 
bodiment, method 200 performs instrument tracking on 
a scrolling series of images using a system or apparatus 
similar to that depicted in FIG. 1 , although other imaging 
systems are contemplated. 

[0047] FIG. 2 illustrates method 200 comprising Step 
210, collecting at least one image. In at least one em- 
bodiment, the collected image may be one or more 2D 
fluoroscopic images, although other images are con- 
templated. In at least one embodiment, method 200 may 
comprise rotating a collection device about or proximate 
a patient's anatomy of interest and collecting one or 
more images using a C-arm and/or detector and/or 
transmitter devices. Method 200 further comprises Step 
220, calibrating at least one image so that a position 
and/or orientation of at least one instrument may be ac- 
curately displayed with such image. In at least one em- 
bodiment, Step 200 may involve image processing tech- 
niques to map image warping that is inherent to many 
C-arms and/or registration techniques that may or may 
not use objects that appear in the at least one image to 
determine the image position within the space of a po- 
sitional measurement device. 

[0048] Embodiments of the present invention further 
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comprise Step 230, computing instrument position and/ 
or orientation within a current image, where an instru- 
ment's position and/or orientation may be projected onto 
or into one or more images. Method 200 further com- 
prises Step 240, displaying at least one collected (i.e., 
current) image and the position/orientation of the at least 
one instrument. 

[0049] In one embodiment, the instrument position 
and/or orientations is projected into each image in se- 
quence. Such scrolling of the data set, including the po- 
sition and/or orientation of the instrument, enables the 
user to u nderstand the instrument position and/or orien- 
tation information in 3D, thus taking advantage of the 
user's natural abilities to perceive 3D information, thus 
eliminating learning new skills to understand the data. 
In addition, at least one embodiment does not require 
direct user interaction to optimize the display as required 
by one or more known methods of the computer assist- 
ed surgical procedures. 

[0050] In at least one embodiment, the instrument 
may be fully displayed on one or more images of the 
data set without partial hiding data due to depth, as is 
required for 3D model instrumentation representations, 
since the depth perception is naturally conveyed to the 
user by the animation and the position information may 
be fully contained within a single visualization window. 
This improves ease-of-use, surgical workflow and prod- 
uct quality perception. 

[0051] FIG. 3 illustrates a detailed flow diagram de- 
picting a method, generally designated 300. for perform- 
ing instrument tracking on a scrolling series of images 
(2D fluoroscopic images for example). In at least one 
embodiment, such method of performing instrument 
tracking uses an imaging system similar to that illustrat- 
ed in FIG. 1 , although other imaging systems are con- 
templated. In at least one embodiment, method 300 
comprises Step 310, rotating a collection device (a C- 
arm for example) about a patient's anatomy of interest. 
Method 300 further comprises Step 320, collecting one 
or more 2D fluoroscopic images during such C-arm and/ 
or detector and/or transmitter devices. 
[0052] Method 300, in at least one embodiment, com- 
prises Step 330, calibrating the collected images such 
that one or more instrument's position and/or orientation 
may be accurately displayed within at least one or more 
images. In one or more embodiments, the instrument 
position and/or orientation may be accurately displayed 
within each such collected image. Fig. 3 further illus- 
trates method 300 comprising Step 340 and 350. Step 
340 comprises selecting a particular image to be a cur- 
rent image. Step 350 comprises computing position 
and/or orientation for at least one instrument within the 
current image. 

[0053] Method 300 further comprises Step 352, pro- 
jecting the instrument position and/or orientation into the 
current image. In one or more embodiments, it is con- 
templated that such instrument position and/or orienta- 
tion may be projected into more than one image in a 



series of images. Step 350 comprises displaying the 
current image in at least one embodiment. In at least 
one embodiment, method 300 further comprises Step 
356 and Step 358 which comprise delaying and incre- 

5 menting a current image, respectively. In at least one 
embodiment, incrementing a current image comprises 
using an algorithm to select a different collected image 
to become the current image. This algorithm may use 
various parameters such as the current direction of trav- 

*0 el through the image sequence, the desired scrolling 
speed and scrolling resolution, and whether the current 
image is at one of the ends of the image sequence to 
determine the next current image. After the current im- 
age is incremented, the instrument position and/or ori- 

'5 entation may be recomputed within the new current im- 
age as illustrated in step 350. 
[0054] For completeness, various aspects of the in- 
vention are set at in the following numbered clauses: 

20 1. A method (200, 300) of performing instrument 
tracking on an image comprising: 

collecting (210, 320) at least one image; 
computing (230. 350) at least one of a position 

25 and orientation of at least one instrument for 

said at least one collected image; and 
displaying (240, 354) at least one of said col- 
lected image, said at least one position and ori- 
entation of said at least one instrument and at 

30 (east one image of said at least one instrument 

located at said at least one of a position and 
orientation. 

2. The method (200, 300) of Clause 1 comprising 
35 collecting (21 0) at least a plurality of 2D fluoroscopic 

images. 

3. The method (200, 300) of Clause 2 comprising 
continuously scrolling through said plurality of said 

40 collected images using a display. 

4. The method (200, 300) of Clause 3 comprising 
projecting (352) said at least one position and ori- 
entation of said at least one instrument into said plu- 

45 rality of collected images in sequence. 

5. The method (200, 300) of Clause 1 comprising 
calibrating (220, 330) said at least one collected im- 
age such that said at least one position and orien- 

50 tation of said at least one image may be accurately 
displayed. 

6. The method (200, 300) of Clause 5 comprising 
selecting (340) at least one calibrated image to be 

55 a current image. 

7. The method (200, 300) of Clause 6 comprising 
computing (230. 350) said at least one position and 
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orientation for said at least one instrument for said 
current image. 

8. The method (200, 300) of Clause 1 comprising 
collecting (320) said at least one image using at 5 
least one moveable collection device. 

9. The method (200, 300) of Clause 8 wherein said 
moveable collection device comprises a C-arm cou- 
pled to an imaging device. 10 

10. A method (300) of performing instrument track- 
ing on a series of images using an imaging device, 
comprising: 

15 

collecting (320) a series of 2D images; 
calibrating (330) said series of 2D images such 
that at least one of a position and orientation of 
at least one instrument may be accurately dis- 
played in at least one image of said series of 20 
images; 

selecting (340) at least one image of said series 
of images to be a current image; 
computing (350) said at least one position and 
orientation of said at least one instrument for 25 
said current image; 

projecting (352) said at least one position and 
orientation within said current image; and 
displaying (354) said current image. 

30 

11 . The method (300) of Clause 1 0 comprising col- 
lecting (320) said series of 2D images using a col- 
lection device that moves. 



18. An apparatus (100) for performing instrument 
tracking on a series of images, the apparatus com- 
prising: 

at least one collection device (120,130) that 
moves and is adapted to collect the set of im- 
ages; 

at least one processing device (1 60) communi- 
cating with at least said moveable collection de- 
vice (1 20, 1 30) and adapted to produce a scroll- 
ing series of images including at least one of a 
position and orientation of at least one instru- 
ment and at least one image of said at least one 
instrument located at said at least one of a po- 
sition and orientation; and 
an output (190) communicating with at least 
said processing device and adapted to display 
at least one image of said scrolling series of im- 
ages. 

19. The apparatus (100) of Clause 18 wherein said 
collection device that moves comprises at least one 
C-arm (110). 

20. The apparatus (1 00) of Clause 1 8 wherein said 
collection device that moves comprises at least 
transmitter (120) and detector (130) devices. 



Claims 

1 . A method (200, 300) of performing instrument track- 
ing on an image comprising: 



12. The method (300) of Clause 11, wherein said 35 
collection device comprises a C-arm coupled to the 
imaging device. 

13. The method (300) of Clause 13 wherein said 
series if 2D images comprise a series of 2D fluoro- *o 
scopic image. 

14. The method (200, 300) Clause 10 comprising 
continually scrolling through said series of images 

in a display. 45 

15. The method (200, 300) of Clause 1 4 comprising 
projecting (352) said at least one position and ori- 
entation of said at least one instrument into at least 
one image of said series of images. so 

16. The method (200, 300) of Clause 1 0 comprising 
incrementing (358) at least said current image. 

1 7. The method (200, 300) of Clause 1 6 comprising 55 
recomputing said at least one position and orienta- 
tion of said at least one instrument. 



collecting (210, 320) at least one image; 
computing (230, 350) at least one of a position 
and orientation of at least one instrument for 
said at least one collected image; and 
displaying (240, 354) at least one of said col- 
lected image, said at least one position and ori- 
entation of said at least one instrument and at 
least one image of said at least one instrument 
located at said at least one of a position and 
orientation. 

2. The method (200, 300) of Claim 1 comprising col- 
lecting (210) at least a plurality of 2D fluoroscopic 
images. 

3. The method (200, 300) of Claim 1 comprising cali- 
brating (220, 330) said at least one collected image 
such that said at least one position and orientation 
of said at least one image may be accurately dis- 
played. 

4. The method (200, 300) of Claim 1 comprising col- 
lecting (320) said at least one image using at least 
one moveable collection device. 
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5. The method {200, 300) of Claim 4 wherein said 
moveable collection device comprises a C-arm cou- 
pled to an imaging device. 

6. A method (300) of performing instrument tracking 5 
on a series of images using an imaging device, 
comprising: 

collecting (320) a series of 2D images; 
calibrating (330) said series of 2D images such 10 
that at least one of a position and orientation of 
at least one instrument may be accurately dis- 
played in at least one image of said series of 
images; 

selecting (340) at least one image of said series 1$ 
of images to be a current image; 
computing (350) said at least one position and 
orientation of said at least one instrument for 
said current image; 

projecting (352) said at least one position and 20 
orientation within said current image; and 
displaying (354) said current image. 

7. The method (300) of Claim 6 comprising collecting 
(320) said series of 2D images using a collection 25 
device that moves. 



The method (200, 300) Claim 6 comprising contin- 
ually scrolling through said series of images in a dis- 
play. 30 

An apparatus (1 00) for performing instrument track- 
ing on a series of images, the apparatus compris- 
ing: 



at least one collection device (120,130) that 
moves and is adapted to collect the set of im- 



35 



at least one processing device (160) communi- 
cating with at least said moveable collection de- 40 
vice (1 20, 1 30) and adapted to produce a scroll- 
ing series of images including at least one of a 
position and orientation of at least one instru- 
ment and at least one image of said at least one 
instrument located at said at least one of a po- 45 
sition and orientation; and 
an output (190) communicating with at least 
said processing device and adapted to display 
at least one image of said scrolling series of im- 
ages. 



10. The apparatus (100) of Claim 9 wherein said collec- 
tion device that moves comprises at least one C- 
arm (110). 
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